Skip to content

Expose OpenAI prompt cache request options - #6832

Open
brandon-julio-t wants to merge 1 commit into
Effect-TS:mainfrom
brandon-julio-t:codex/6830-openai-request-cache-options
Open

Expose OpenAI prompt cache request options#6832
brandon-julio-t wants to merge 1 commit into
Effect-TS:mainfrom
brandon-julio-t:codex/6830-openai-request-cache-options

Conversation

@brandon-julio-t

Copy link
Copy Markdown
Contributor

RFC status

Implementation complete and tested. This PR is ready for review. It is framed as an RFC because feedback is requested on the public OpenAiSchema.CreateResponse / OpenAiLanguageModel.Config shape, not because implementation work remains.

Summary

Expose the following OpenAI Responses API request-level options through OpenAiLanguageModel.Config:

  • safety_identifier
  • prompt_cache_key
  • legacy prompt_cache_retention (in_memory or 24h)
  • GPT-5.6+ prompt_cache_options (mode and ttl)

The request builder already forwards schema-derived config, so the implementation is deliberately limited to the handwritten request schema, request/schema tests, and a patch changeset. Content-level prompt_cache_breakpoint is not included; that remains #6831.

Design decisions

  • Keep request configuration in the handwritten OpenAiSchema.CreateResponse, which is the source for the high-level model config and client request type.
  • Model prompt_cache_options.mode as optional implicit | explicit and ttl as optional 30m, matching current GPT-5.6 Responses request semantics and defaults.
  • Retain the legacy pre-GPT-5.6 field separately as prompt_cache_retention: in_memory | 24h.
  • Enforce the documented 64-character maximum for safety_identifier.
  • Do not add model-name gating. The model type accepts custom strings, so compatibility remains an OpenAI API concern rather than a brittle client-side model registry.
  • Leave @effect/ai-openai-compat unchanged: it uses a separate Chat Completions-compatible request surface and is not the Responses high-level config reported in this issue.

Generated vs handwritten schema

The checked-in generated OpenAI request schema already contains safety_identifier, prompt_cache_key, and prompt_cache_retention, but it does not contain the newer prompt_cache_options. Current official OpenAI Responses documentation and the official Node SDK do document prompt_cache_options for GPT-5.6+.

This PR does not hand-edit Generated.ts or add a speculative codegen patch. The generated client is not the source of OpenAiLanguageModel.Config; keeping this change in the handwritten Responses schema provides the requested high-level support without creating a broad generated diff. Generated parity can follow when the configured Stainless schema includes the field, or via a maintainer-preferred codegen patch.

Focused RFC questions

  1. Should prompt_cache_options remain an inline nested schema, consistent with nearby request options, or be exported as a named public schema?
  2. Should these high-level optional config fields also accept explicit null to mirror raw OpenAI SDK types, or remain optional non-null values consistent with the existing handwritten config style?
  3. Is leaving Generated.ts unchanged preferable until the configured upstream schema catches up, or would maintainers prefer a targeted codegen patch now?

Verification

  • pnpm lint-fix
  • pnpm test --run packages/ai/openai/test/OpenAiLanguageModel.test.ts packages/ai/openai/test/OpenAiSchema.test.ts — 61 tests passed
  • pnpm check
  • Independent review loop completed; the final pass reported no actionable findings.

Positive tests verify that all supported fields survive high-level model config and reach real HTTP request bodies:

  • non-streamed GPT-5.5 request: safety identifier, cache key, and legacy retention
  • streamed GPT-5.6 request: safety identifier, cache key, cache options, and stream: true

Schema coverage also verifies the supported option shapes and rejects safety identifiers longer than 64 characters.

Tradeoffs

  • The strict enums intentionally track current documented values; future OpenAI values will require a schema update.
  • No client-side model compatibility check is added, avoiding false rejection of custom or newly released model identifiers.
  • Generated and handwritten request schemas remain temporarily asymmetric for prompt_cache_options; that limitation is explicit rather than hidden by guessing at generated output.

Closes #6830

@changeset-bot

changeset-bot Bot commented Aug 1, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 7b0852e

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 30 packages
Name Type
@effect/ai-openai Patch
effect Patch
@effect/opentelemetry Patch
@effect/platform-browser Patch
@effect/platform-bun Patch
@effect/platform-deno Patch
@effect/platform-node-shared Patch
@effect/platform-node Patch
@effect/vitest Patch
@effect/ai-anthropic Patch
@effect/ai-openai-compat Patch
@effect/ai-openrouter Patch
@effect/atom-react Patch
@effect/atom-solid Patch
@effect/atom-vue Patch
@effect/sql-clickhouse Patch
@effect/sql-d1 Patch
@effect/sql-libsql Patch
@effect/sql-mssql Patch
@effect/sql-mysql2 Patch
@effect/sql-pg Patch
@effect/sql-pglite Patch
@effect/sql-sqlite-bun Patch
@effect/sql-sqlite-do Patch
@effect/sql-sqlite-node Patch
@effect/sql-sqlite-react-native Patch
@effect/sql-sqlite-wasm Patch
@effect/docgen Patch
@effect/doctest Patch
@effect/openapi-generator Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

  • New CreateResponse fields — added safety_identifier (64-char max), prompt_cache_key, prompt_cache_retention ("in_memory" | "24h"), and prompt_cache_options ({ mode?: "implicit" | "explicit", ttl?: "30m" }) to the handwritten Responses schema. These flow automatically into Config via the existing Partial<Omit<CreateResponse.Encoded, ...>> type and through to HTTP request bodies via the ...apiConfig spread in makeRequest.
  • Integration tests — one non-streamed test (GPT-5.5, legacy retention) and one streamed test (GPT-5.6, new cache options) verify all fields reach HTTP request bodies. The streaming test uses the new makeStreamHttpTestLayer which exercises the real OpenAiClient and HTTP pipeline, a better integration path than the existing client-mock makeStreamTestLayer.
  • Schema test — validates decode of all four fields and rejects safety_identifier over 64 characters.
  • Test infrastructureMockOpenAiResponse gained an optional events field (backwards-compatible — body was always provided in existing calls) and makeHttpClient routes to SSE-format responses when events are present.

Pullfrog  | View workflow run | Using DeepSeek Pro (free via Pullfrog for OSS) (Kimi K2 not used — the program covers this model; add its provider key to run your pick) | 𝕏

@github-actions

github-actions Bot commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Bundle Size Analysis

Generated from PR build output; treat the content below as untrusted.

File Name Current Size Previous Size Difference
basic.ts 6.63 KB 6.63 KB 0.00 KB (0.00%)
batching.ts 9.42 KB 9.42 KB 0.00 KB (0.00%)
brand.ts 6.31 KB 6.31 KB 0.00 KB (0.00%)
cache.ts 10.16 KB 10.16 KB 0.00 KB (0.00%)
config.ts 20.32 KB 20.32 KB 0.00 KB (0.00%)
differ.ts 19.93 KB 19.93 KB 0.00 KB (0.00%)
http-client.ts 21.02 KB 21.02 KB 0.00 KB (0.00%)
logger.ts 10.32 KB 10.32 KB 0.00 KB (0.00%)
metric.ts 8.55 KB 8.55 KB 0.00 KB (0.00%)
optic.ts 7.33 KB 7.33 KB 0.00 KB (0.00%)
pubsub.ts 14.46 KB 14.46 KB 0.00 KB (0.00%)
queue.ts 11.13 KB 11.13 KB 0.00 KB (0.00%)
schedule.ts 10.31 KB 10.31 KB 0.00 KB (0.00%)
schema-class.ts 18.86 KB 18.86 KB 0.00 KB (0.00%)
schema-fromJsonSchemaDocument.ts 28.67 KB 28.67 KB 0.00 KB (0.00%)
schema-representation-roundtrip.ts 24.99 KB 24.99 KB 0.00 KB (0.00%)
schema-string-transformation.ts 12.99 KB 12.99 KB 0.00 KB (0.00%)
schema-string.ts 10.65 KB 10.65 KB 0.00 KB (0.00%)
schema-template-literal.ts 14.85 KB 14.85 KB 0.00 KB (0.00%)
schema-toArbitraryLazy.ts 21.66 KB 21.66 KB 0.00 KB (0.00%)
schema-toCodeDocument.ts 24.08 KB 24.08 KB 0.00 KB (0.00%)
schema-toCodecJson.ts 18.91 KB 18.91 KB 0.00 KB (0.00%)
schema-toEquivalence.ts 18.73 KB 18.73 KB 0.00 KB (0.00%)
schema-toFormatter.ts 18.59 KB 18.59 KB 0.00 KB (0.00%)
schema-toJsonSchemaDocument.ts 22.33 KB 22.33 KB 0.00 KB (0.00%)
schema-toRepresentation.ts 19.25 KB 19.25 KB 0.00 KB (0.00%)
schema.ts 18.12 KB 18.12 KB 0.00 KB (0.00%)
stm.ts 12.11 KB 12.11 KB 0.00 KB (0.00%)
stream.ts 9.37 KB 9.37 KB 0.00 KB (0.00%)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

4.0 enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Expose OpenAI Responses safety and prompt-cache request options through OpenAiLanguageModel.Config

1 participant